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Abstract 

The  use  of  repeated  measures  designs  in  many  areas  of  psychological 
research  has  prompted  concern  for  the  potential  confounds  inherent 
in  the  Interpretation  of  within-subjects  effects.  Of  the  solutions 
proposed  for  this  problem,  the  most  commonly  adopted  strategy  is 
the  use  of  Latin  Square  counterbalancing  orders  for  treatment 
presentation.  Traditional  Latin  Square  designs  ensure  that  each 
of  the  experimental  treatments  appear  equally  often  in  each  serial 
position  of  the  treatment  sequence.  The  present  paper  presents  a 
technique  for  generating  a  subset  of  Latin  Squares  that  control 
for  two  additional  characteristics  of  treatment  sequence.  Pairwise 
priority  refers  to  the  proportion  of  times  that  for  any  given 
treatment  pair,  x  and  y_.  Treatment  x  precedes  Treatment  y_.  A 
subset  of  Latin  Squares  exists  for  which  this  proportion  is  .5  for 
all  treatment  pairs.  Pairwise  distance  refers  to  the  number  of 
other  treatments  that  come  between  treatment  pair  x  and  in  the 
treatment  sequence.  A  subset  of  Latin  Squares  exists  that  partially 
controls  for  the  distribution  of  distances  across  all  treatment 
pairs.  The  subset  of  Latin  Squares  that  controls  for  both 
pairwise  priority  and  pairwise  distance  are  referred  to  as  'pairwise  ■ 
balanced*  Latin  Squares. 
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Within-subject  designs  are  being  used  with  increasing  frequency 
in  psychological  research.  For  example,  Poulton  (1982)  compared  the 
types  of  experimental  designs  employed  In  research  reported  in  the 
Journal  of  Experimental  Psychology  in  September  1972  and  1979  and 
found  that  the  ratio  of  within-subject  to  between-subjects  designs 
had  increased  from  1.7:  1  to  7.3:  1.  This  increase  has  been 

accompanied  by  commentary  and  ana  Vs  is  concerning  the  adequacy  of 
such  within-subject  designs  to  provide  unambiguous  tests  of 
experimental  hypotheses  (e.g.,  Greenwald,  1976;  Poulton,  1973, 

1974,  1982;  Sothsteln,  1974).  The  central  concerns  embodied  in 
these  commentaries  relate  both  to  matters  of  experimental  procedure 
and  the  proper  interpretation  of  experimental  results.  Of  course, 
the  two  are  Interrelated  in  that  improvements  in  procedure  often 
Serve  to  lessen  interpretive  cautions. 

The  most  efficient  procedure  for  dealing  with  the  Interpretive 
problems  of  within-subject  designs  involves  the  use  of  Latin  Square 
counter-balanced  orders  for  treatment  presentation  (e.g.,  Lindman, 
1974;  Myers,  1979;  Winer,  1972).  The  purpose  of  the  present  paper 
is  to  address  the  adequacy  of  traditional  Latin  Square  selection 
criteria.  The  traditional  criteria  focus  exclusively  on  guaranteeing 
that  all  treatments  appear  equally  often  in  all  serial  order  positions 
of  the  treatment  presentation  sequence.  In  this  paper,  we  argue 
that  two  additional  criteria  should  always  be  invoked  when  selecting 
a  Latin  Square,  namely  the  criteria  of  "pairwise  priority"  and 
"pairwise  distance". 
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Within-subj ect  Treatments  and  Latin  Square  Designs 

In  within-subj ect  designs,  each  subject  is  exposed  to  all 
of  the  experimental  treatments.  This  type  of  design  is  often 
preferred  because:  a)  it  allows  for  greater  economy  in  subject 
utilization,  b)  it  often  serves  to  increase  the  statistical  power 
of  hypothesis  tests,  and  c)  it  is  often  a  more  ecologically  valid 
way  of  examining  specific  research  hypotheses  (e.g.,  Greenwald, 

1976,  but  see  Poulton,  1982). 

Although  preferred  for  these  reasons,  within-subj ect  designs 
are  also  encumbered  by  procedural  weaknesses  that  often  leave  the 
research  open  to  plausible  alternative  explanations  for  the  obtained 
experimental  results.  Chief  among  these  potential  confounds  are 
those  associated  with  order  or  sequence  effects,  practice  and/or 
fatique  effects,  and  the  residual  effects  (also  referred  to  as 
transfer,  carry-over  or  range  effects)  of  treatments  (e.g., 

Campbell  &  Stanley,  1966;  Carlamith,  Ellsworth  &  Aronson,  1976; 
Christensen,  1980;  Crano  &  Brewer,  1973;  D'Amato,  1970;  Greenwald, 
1976;  Poulton,  1973,  1974,  1982).  Several  general  types  of 
solutions  have  been  suggested  in  attempts  to  take  such  potential 
sequence  effects  into  account.  The  most  commonly  adopted  procedure 
involves  the  use  of  Latin  Square  counterbalancing  of  treatment 
orders . 

Traditional  criteria  for  Latin  Square  selection. 

Latin  Squares  control  serial  position  effects  by  ensuring 
that  each  treatment  appears  equally  often  in  each  order  position. 
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Traditional  selection  criteria  focus  on  random  selection  from  the 
population  of  all  possible  squares.  Consider  the  guidelines  outlined 
by  Winer  (1972).  He  suggests  that  one  first  randomly  select  a 
standard  square  from  such  sources  as  Fisher  &  Yates  (1953)  or 
Cochran  &  Cox  (1957).  Next,  the  columns  and  rows  are  randomly 
reordered.  Winer  provides  an  example  for  the  4X4  Latin  Square 
case.  Starting  with  the  square  on  the  left  of  Table  1,  Winer 
reordered  the  colunns  and  rows  according  to  the  random  number 
sequences  2,  4,  1,  3  and  3,  4,  1,  2  producing  the  square  on  the 


Insert  Table  1  about  here 

right  of  Table  1.  In  Latin  Square  designs  such  as  this,  the 
columns  refer  to  the  serial  order  of  treatments  (a  within -subjects 
factor)  and  the  rows  refer  to  subject  types  (a  between-subjects 
factor) . 

In  practice,  many  Investigators  bypass  the  recommended 
procedure  and  generate  their  own  square  in  the  simplest  manner 
possible.  This  can  be  done  by  randomly  assigning  treatments  to 
positions  in  the  first  row  of  a  square  and  then  cyclically  permuting 
each  Mbsequent  row.  To  do  this,  one  simply  takes  the  last  condition 
of  the  first  row  and  puts  it  in  the  first  position  of  the  second  row. 
All  other  treatments  are  then  shifted  accordingly  one  position  to 
the  right.  By  coincidence,  the  recommended  square  produced  by 
Winer's  randomisation  procedure  (see  Table  1)!  yielded  such  a 
cyclical-  square.  This  can  be  seen  most  easily  by  transposing  rows 
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two  and  three  in  Winer's  reconmended  square.  Even  Cochran  and  Cox 
(1957,  p.  145-146)  revert  to  the  use  of  cyclical  squares  when  n 
la  greater  than  6.  Such  cyclical  squares,  unfortunately,  always 
introduce  pairwise  biases,  and  therefore  should  always  be  avoided 
for  counterbalancing  in  repeated  measures  designs. 

New  Criteria  for  Latin  Square  Selection 

It  is  clear  that  the  cyclically  generated  squares,  as  well  as 
those  generated  in  the  traditional  manner,  all  satisfy  the  selection 
criterion  of  ensuring  that  each  treatment  appears  in  each  of  the 
four  treatment  serial  positions.  These  commonly  used  squares, 
however,  fail  to  explicitly  control  for  two  other  features  of 
treatment  sequence  that  can  affect  the  interpretation  of  within- 
subject  treatment  differences,  i.e.,  pairwise  priority  and  pairwise 
distance. 

Pairwise  priority  refers  to  the  proportion  of  times  (across 
all  subject  types)  that  "treatment  x"  precedes  "treatment 
When  that  proportion  is  exactly  .5,  this  means  that  x  precedes  £ 
as  often  as  £  precedes  x.  For  example,  note  that  in  the  condition  pair  of 
0,  1  in  the  recommended  square  of  Table  1  the  proportion  is  exactly 
.5  (or  2/4),  whereas  for  the  pair  0,  2  the  proportion  is  .75  (or 
3/4) .  A  subset  of  squares  exists  in  which  all  pairs  have  exactly 
a  .5  probability.  Such  squares  are  considered  to  be  balanced  for 
pairwise  priority. 

Pairwise  distance  refers  to  the  number  of  other  treatments 
(counting  forward  or  backward  from  the  numerically  smaller  member 
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of  the  treatment  pair  to  the  larger)  occurring  between  a  particular 
pair  of  treatments,  x  and  For  example,  in  the  first  line  of  the 
recommended  square  of  Table  1,  there  is  a  distance  of  two  units 
between  the  1,  3  condition  pair.  Ideally,  one  would  want  to 
exactly  control  the  distribution  of  distances  over  the  entire 
design  for  all  condition  pairs.  Unfortunately  no  Latin  Squares 
exist  that  provide  such  a  control. 

There  are  two  features  of  the  distributions  of  pairwise 
distances  that  can  be  controlled  within  a  single  square.  The 
first  is  the  proportion  of  pairs  that  are  contiguous  (i.e. ,  the 
proportion  of  times,  over  all  subject  types,  that  a  particular 
pair  has  a  distance  of  zero) .  Note  in  the  recommended  square  of 
Table  1  the  pair  0,  2  are  contiguous  three  of  four  times,  whereas 
the  pair  0,  1  are  never  contiguous.  Of  the  six  pairs  in  this 
square,  four  have  at  least  one  contiguous  occurrence  and  two 
(0,  1  and  2,  3)  have  no  contiguous  subject  types.  A  subset  of 
Latin  Squares  exists  in  which  all  condition  pairs  have  exactly 
two  subject  types  with  zero  distance.  In  such  squares ,  the 
proportion  of  contiguous  pairs  is  constant  for  all  possible  pairs. 

There  is  a  second  feature  of  the  distribution  of  distances 
between  pairs  that  can  be  controlled.  In  squares  balanced  for 
pairwise  priority,  it  is  possible  to  obtain  directional  symmetry. 

One  can  exactly  match  the  distribution  of  distances  for  subject 
types  in  which  Condition  x  precedes  Condition  ^  with  the  distribution 
obtained  w>  Con'  .ion  x  follows  Condition  £.  Thus  directional 
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differences  in  priority  will  not  be  confounded  with  distance.  Latin 
Squares  that  control  for  these  two  features  of  distance  (directional 
symmetry  and  proportion  of  contiguous  pairs)  are  considered  balanced 
for  distance. 

Construction  of  Pairwise  Balanced  Latin  Squares 
The  existence  of  subsets  of  Latin  Squares  incorporting  features 
related  to  the  present  concerns  has  been  acknowledged  in  the  past 
(e.g.,  "diagram"  balanced  designs  of  Wagenaar,  1969,  and  designs 
"balanced  for  the  estimation  of  residual  effects",  as  discussed  by 
Allmena,  1962;  Cochran  &  Cox,  1959;  and  Williams,  1949).  However, 
these  earlier  authors  have  not  addressed  the  special  implications 
of  these  squares  for  counterbalancing  in  psychological  research. 

We  have  also  been  able  to  improve  on  the  procedures  presented  in 
this  earlier  work  for  identifying  acceptable  squares  (Isaac,  McCann 
&  Ostrom,  1983) .  For  example,  our  procedure  for  even  number  designs 
generates  more  squares  than  does  Williams'  (1949)  procedure,  and 
Includes  squares  equivalent  to  those  generated  by  Allmena  (1962) 
and  Wagenaar  (1969).  Reports  of  these  earlier  procedures  are  absent 
from  many  books  on  statistics  (e.g.,  Winer,  1972;  Meyers,  1979)  and 
research  design  (e.g.,  Crano  &  Brewer,  1973;  Murphy  &  Puff,  1982) 
that  appear  in  the  psychological  literature. 

Since  procedures  for  generating  pairwise  balanced  pairs  are 
available  elsewhere,  we  will  not  repeat  them  here.  Instead  we 
have  prepared  tables  that  summarize  squares  ranging  in  size  from 
three  to  sixteen.  Most  repeated  measures  research  in  psychology 


-  *’«  ’  «  •  •  *  »  "  Su' 


’  • '  v*  «_**  • 

■■■■■■•-vWv 


Pairwise  Balanced  Latin  Squares 

10 

involves  designs  in  that  range.  We  should  also  note  at  this  point 
that  no  single  generation  procedure  exhaustively  represents  the 
entire  population  of  pairwise  balanced  squares  for  any  given  n  (see 
Isaac,  McCann,  &  Ostrom,  1983). 

Insert  Tables  2  and  3  about  here 

The  entries  in  Tables  2  resulted  from  applying  procedures  referred 
to  in  Isaac  et  al.  (1983).  Those  in  Table  3  were  initially  developed 
by  Williams  (1949).  We  suspect  that  even  more  could  be  produced 
through  trial  and  error  (see  Wagenaar,  1969,  and  Denes  &  Keedwell, 
1974). 

How  to  use  Tables  2  and  3 

Tables  2  and  3  do  not  contain  the  full  Latin  Squares;  rather, 
they  provide  only  the  first  line  of  the  one  or  more  pairwise  balanced 
squares  given  for  each  size  n.  This  first  line  corresponds  to 
"Subject  Type  I"  as  described  in  Table  1.  The  lines  corresponding 
to  the  remaining  Subject  Types  are  easily  produced  in  the  manner 
described  below. 

1.  Determine  the  size  of  Latin  Square  needed  for  the 
research  design.  The  size  (n)  corresponds  to  the  number  of  treatments 
in  the  repeated  measures  experiment . 

2.  Select  a  first  line  from  Table  2  or  3  that  corresponds 

to  n.  If  more  than  one  is  listed  In  the  table,  select  one  randomly. 

3.  Generate  the  remaining  rows  (or  Subject  Types)  of  the 
square.  Successive  rows  are  produced  by  adding  one  (in  modular 
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arithmetic)  to  each  entry  in  the  previous  row.  As  an  illustration, 
consider  the  square  of  n  *»  4  in  Table  2.  The  second,  third,  and 
fourth  rows  are  1,  2,  0,  3;  2,  3,  1,  0;  and  3,  0,  2,  1,  respectively. 

4.  Randomly  assign  experimental  treatments  to  the  n  numbers 
in  the  resulting  square.  Note  that  this  means  when  an  n  by  n  square 
is  generated,  it  forms  the  basis  of  n!  squares  of  experimental 
treatments. 

5.  Randomly  assign  subjects  to  rows  of  the  square,  insuring 
that  an  equal  number  of  subjects  are  assigned  to  each. 

Odd  size  squares 

Complications  arise  when  an  experiment  involves  an  odd  number 
of  experimental  conditions.  Whereas  complete  pairwise  balance 
can  be  achieved  with  a  single  square  for  n  even,  this  cannot  be 
done  in  the  case  of  n  odd.  For  example,  it  is  impossible  to 
achieve  pairwise  priority  for  ii  odd  since  the  proportion  of  times 
Condition  x  precedes  Condition  y  can  never  be  exactly  .5.  In  this 
case,  two  squares  must  be  used  to  achieve  design -wide  pairwise 
balance.  This  can  be  done  by  selecting  any  first  row  from  Table  3 
and  combining  it  with  a  square  based  on  the  reverse  of  the  selected 
first  row. 

One  implication  of  using  two  squares  for  n  odd  is  that  the 
minimum  number  of  subjects  required  for  full  counterbalancing 
increases  from  n  to  2n.  This  suggests  that  there  is  a  distinct 
advantage  to  employing  repeated  measures  designs  in  which  the  total 


number  of  conditions  is  even.  Thus,  if  the  minimum  number  of 
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conditions  needed  to  test  the  experimental  hypothesis  is  odd,  the 
researcher  is  urged  to  consider  the  benefits  of  adding  one 
theoretically  relevant  condition.  This  would  result  in  increased 
economy  in  terms  of  the  minimum  number  to  subjects  required. 

When  the  main  experimental  concern  is  with  trends  over  a 
parametric  independent  variable  (e.g.,  set  size  or  exposure  time), 
adding  one  more  condition  will  also  allow  for  a  test  of  an  additional, 
higher  order  orthogonal  polynomial.  When  the  repeated  measures  are 
the  result  of  a  factorial  design  (e.g.,  set  size  by  exposure  time 
by  familiarity  of  word  type) ,  it  is  necessary  that  only  one  factor 
have  an  even  number  of  levels.  Conditions  under  which  odd  squares 
should  still  be  used  occur  when  an  increase  to  an  even  design  would 
result  in  excessive  expense  or  excessive  running  time  for  subjects. 

Statistical  Considerations 

Designs  reported  in  this  paper  are  balanced  for  additive  residual 
or  carry-over  effects  of  the  immediately  preceding  treatment.  It 
should  be  noted  that  the  residual  effects  may  be  more  complicated; 
for  example,  multiplicative  effects  or  those  persisting  beyond  the 
immediately  preceding  treatment.  If  the  structure  of  the  residual 
is  of  some  more  complicated  sort,  these  designs  or  any  other  Latin 
Squares  may  not  be  appropriate.  The  investigation  of  such  residual 
effects  is  beyond  the  scope  of  the  present  paper. 

Latin  square  designs  were  originally  developed  to  deal  with 
residual  effects  that  are  additive.  To  this  point,  the  primary 
concern  has  been  to  control  for  the  additive  effects  of  serial 
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position.  Conventional  statistical  analyses  routinely  include 
tests  of  significance  for  this  factor.  (It  should  be  noted  that 
these  .  serial  position  effects  are  usually  not  of  interest  in 
psychological  research  and  in  practice  are  rarely  even  reported.) 

A  unique  feature  of  pairwise  balanced  Latin  Squares  is  that  they 
make  it  possible  to  statistically  estimate  the  contribution  of 
additive  residual  effects  due  to  pairwise  priority.  To  do  this, 
one  must  use  an  analysis  proposed  by  Williams  (1949)  and 
illustrated  in  Cochran  and  Cox  (1957). 

Perhaps  obviously, a  standard  statistical  analysis  of  a  Latin 
Square  could  be  used  in  the  present  case,  assuming  pairwise  residual 
effects  exist,  and  that  treatments  and  residual  effects  are 
wcorrelated,  then  there  will  be  a  positive  bias  in  the  mean  square 
for  treatments  (i.e.,  the  mean  square  will  be  larger  than  in  such 
residual  effects  didn't  exist),  and  estimates  of  treatment  effects 
will  be  confounded  with  residual  effects.  However,  since  the 
design  is  balanced,  treatment  effects  will  be  confounded  with  their 
own  residual  effects.  In  contrast,  Latin  Square  designs  that  are 
not  pairwise  balanced  will  have  treatment  effects  which  may 
be  confounded  with  residual  effects  of  other  treatments.  Further, 
there  will  also  be  a  positive  bias  in  the  mean  square  for  treatments 
the  extent  of  which  will  depend  on  the  particular  design,  but  which 
in  general  will  be  greater  than  that  associated  with  the  pairwise 
balanced  designs. 

Thus,  if  it  is  desirable  to  estimate  the  test  direct  treatment 
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effects  and  residual  effects  separately,  the  analysis  given  in 
Cochran  and  Cox  (1957)  is  recommended.  Note  that  this  analysis 
applies  strictly  to  pairwise  balanced  designs,  and  not  to  other 
designs  in  which  carry-over  may  be  suspected.  Alternatively, 
a  standard  analysis  of  Latin  Square  designs,  such  as  given  in 
most  textbooks,  would  be  testing,  in  effect,  the  significance 
of  an  additive  combination  of  treatments  and  their  residual 
effects  when  applied  to  the  pairwise  balanced  designs. 
Computational  Procedures 

The  computational  procedures  for  pairwise  balanced  designs 
were  first  described  by  Williams  (1949,  1950),  and  later  reported 
in  slightly  modified  form  by  Cochran  &  Cox  (1957) .  Since 
neither  source  is  commonly  available  to  psychologists,  we  will 
present  the  Cochran  &  Cox  notation,  and  illustrate  its  use  with 
an  example. 

Designs  differ  in  terms  of  whether  more  than  one  square  is 
used  and  whether  more  than  one  subject  is  assigned  to  each  row 
of  the  square.  Normally,  two  or  more  squares  will  be  used  when 
jn  is  odd.  But  also,  it  will  sometimes  be  advantageous  to  use 
several  squares  in  the  case  of  n  even.  It  will  increase  error 
degrees  of  freedom  and  can  allow  for  greater  control  over  the 

Insert  Table  4  about  here 

distribution  of  pairwise  distances.  We  have  selected  an  example 
employing  two  3x3  squares  with  one  subject  per  row. 
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The  analyses  presented  here  assune  that  row,  column,  and 
treatment  effects  do  not  interact,  and  further,  that  the  residual 
effect  simply  adds  to  the  effect  of  the  following  treatment. 

Thus,  for  a  row  in  which  B  follows  A,  and  C  follows  B  (i.e., 
treatment  order  ABC),  the  period  in  which  C  is  applied  has 
a  total  effect  attributable  to  treatments  which  is  (tc  +  rfe) . 
Similarly,  the  total  observed  effect  of  treatments  when  B  is 
applied  would  be  (tjj  +  ra)  . 

To  simplify  the  presentation,  let. us  assume  that  an  experiment 
involves  responses  to  three  attitude  statements.  A,  B,  C,  all 
presented  to  each  subject.  A  given  subject  responds  to  a  row 
of  an  appropriately  selected  square,  and  a  column  corresponds 
to  the  position  in  the  order  of  presentation.  Two  squares  are 
selected  to  be  pairwise  balanced.  The  squares  are  given  in 
Table  4.  Included  in  Table  4  is  the  hypothetical  data,  with  one 
observation  per  cell. 

The  following  symbols  are  used: 
n  -  number  of  treatments  («=3) 
m  -  number  of  squares  («2) 

£  ■  total  of  sequence  (row) 

T  -  treatment  total 

P. •  total  of  position  (column)  in  a  given  square 
R  ■  total  of  scores  in  positions  immediately  following  the 
treatment  in  question 

F  ■  total  of  sequences  (rows)  in  which  this  treatment  is  the 


last  one 


o?  ol 
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-  total  of  scores  in  first  postion  of  all  sequences 
“  grand  total  of  scores  in  a  given  square 
•  grand  total  of  all  scores 
The  following  quantities  special  to  this  analysis  are  needed 
P,  -  nG 

nP 1  -  (n  +  2)G 

Then  some  of  the  usual  quantities  are  needed: 

Correction  factor:  C  -  9?- 

mnz 

Total  SS  -  EX2  -  C  (df  -  an2  -  1) 

Sequences  SS  -  i.zg2  _  c  (df  -  mn  -  1) 
n 

Positions  (Order)  SS  -  ilP2  -  —  EG. 2  (df  =  m(n  -  1)) 

n  —  n  —J 

Note  that  this  is  a  sum  of  squares  between  positions  within 
squares . 

Treatments  (unadjusted)  SS  *  J!!L£  _  c  “2.-1) 


4.' !!' "«  W  U l' «.* <■ 


VT  <  ».  w 
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In  addition,  adjusted  (for  residual  or  carry-over)  treatment 
effects  are  computed  as  follows: 

T  •  (n2  -  n  -  1)  T  +  nR  +  F  +  P  -  nG 

and  the  adjusted  sum  of  squares  for  treatments  is  computed: 

ss  ,  _ IT} _  (df  ■  n  -  1) 

trts(adj)  mn  (n2  -  n  -  1) (n2  -  n  -  2) 

Similarly,  adjusted  residual  (carry-over)  effects  are  computed: 

R  -  nT  +  n2R  +  nF  +  nPi  -  (n  +  2)G 
and 


SSres (ad j ) 


Z$2 _ 

mn3(n2  -  n  -  2) 


The  total  sum  of  squares  for  treatment  effects  (i.e.,  direct 
plus  residual)  is  given  by: 

SStrt  “  SStrts(unadj)  +  SSres(adj) 
or  -  SStrt8(adj)  +  SSreg(unad;J) 

Three  of  these  four  quantities  were  computed  above;  the  fourth, 

^res(unadj)  ma^  ^e  comPute<*  by  subtraction.  However,  to  provide 
a  check  on  computations,  SSrgg^unadj^  may  be  computed  directly: 
First,  for  each  treatment  compute 
R'  ■  ft  +  G  -  nT 


4 


1 


i 


l 


S  .** 

a  v 


>  ,'.;v  ,v  •• .v  A 
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Then  the  sum  of  squares  is  given  by 
SSr.««n.dD  ’  .  l) 

Finally,  SS  error  is  obtained  by  subtraction  from  the  total 
sum  of  squares.  Here  the  sum  of  squares  for  Positions  within 
Squares  has  been  removed.  If  instead  the  overall  sum  of  squares 
for  Positions  Is  removed  (i.e.,  assuming  no  differences  in  Positions 
effects  across  squares),  the  error  degrees  of  freedom  becomes 
<n  -  1) (an  -  3). 

Insert  Tables  5  and  6  about  here 

Table  5  summarizes  the  computations  using  the  formulae 
that  are  presented  above.  The  analysis  of  Variance  Summary  Table 
is  contained  in  Table  6.  Note  that  the  terms  in  brackets  have 
the  same  total.  Tests  on  the  effects  of  treatments  and  residuals 
should  only  be  made  on  the  adjusted  sums  of  squares.  The  Error 
sum  of  squares  is  obtained  by  subtraction  of  the  total  of  all 
other  non-redundant  sums  of  squares  from  the  Total,  i.e.; 

SStQt  -  182.0,  Thus,  only  one  of  the  sums  in  brackets  is 
unsolved  in  this  subtraction,  otherwise  effects  attributable  to 
treatments  (direct  and  residual)  would  be  counted  twice.  Since 
Fj  ^  *  10.65  for  a  -  .025,  both  direct  treatment  and  residual 
effects  are  significant  in  this  example. 
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This  example  does  not  Involve  a  design  in  which  a  between  Ss 
treatment  effect  is  included,  or  in  which  multiple  Ss  are  assigned 
to  a  row  of  a  square.  However,  the  extent ion  of  the  analysis  to 
such  designs  is  straightforward  and  follows  plans  available  in 
standard  texts.  The  adjustment  for  carry-over  (residual)  involves 
adjustment  of  within  subject  effects.  Further,  computation  of  a 
Ss  x  Positions  effect  (the  usual  within  J5s  error)  in  this  case 
would  be  unaffected.  Between  Ss  effects  would  use  the  usual 
Between  Ss  error  term. 
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Table  1 

Latin  Square  Selection  Based  on  Traditional  Criteria 


Standard  Square  Recommended  Square 

Treatment  Order  Treatment  Order 

Subject  Type  First  Second  Third  Fourth  First  Second  Third  Fourth 


2  3 
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Table  4 

Pairwise  Balanced  3x3  Latin  Squares  and  Simulated  Data 

Latin  Squares 

Square  1  Square  2 

Treatment  Position  Treatment  Position 

Subject  Type  First  Second  Third _ Subject  Type  First  Second  Third 


I 

II 

III 


0 

1 

2 


1  2 
2  0 
0  1 


IV 

V 

VI 


2 

0 

1 


1  0 
2  1 
0  2 


Simulated  Data 


-4 

1 

3 


Square  1 

-6  1 

1  0 

2  -1 


£ 

-9 

2 

4 


6 

-4 

0 


Square  2 

7  -2 

-2  2 

1  1 


£ 

11 

-4 

2 


-3 


-3 
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Table  5 

Summary  of  Computations 


Number 

T 

R 

F 

i 

t=T/24 

R 

r=R/24 

K' 

0 

-7 

-8 

13 

-62 

-2.58 

178 

-3.25 

-51 

1 

3 

1 

0 

2 

.08 

-6 

-.25 

-9 

2 

10 

11 

-7 

60 

2.50 

84 

3.5 

60 

P x  -  nG  -  2  -  3(6)  -  -16 

nP1  -  (n  +  2)G  -  3(2)  -  (3  +2)6  -  -24 

f  -  5T  +  3R  +  F  -  16 

R'  -  3T  +  9R  +  3F  -  24 


„  _  9.  _  36  _  2.n 

-  2  •  37  18 


sstot  ■  2x2  - 


C  -  184  -  2  -  182 


ss8eq  "  3"  {Z-2}  '  -  "  j((-9)2  +  (22)  +  (2)2}  -  2  -  78.67 
SSpOB  *'***  "  J{ZI2}  * 


J  io2  +  (-3)2  +  ...  +  (l)2}  -  |{(-3)2  +  (9)2) 


trt(unadj)  -  g 


3L  -  c  -  7<(-7)2  +  (3)2  +  (10)2}  -  2  =  24.33 


VfV 
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P 

k 


m 


■««»  •6-^r  •  (-62>2  \{f  * (60)2-  62-07 


SS 


ZR2 


res (ad j )  ~2*33  (3*  -  3  -  2) 


(-78) 2  +  (-6) 2  +  (84) 2  *  61.00 


216 


SSres(unadj)  "  —3-^  - - -  *  +  <-9)2_t IM2  =  23.27 

J  2-3°  (3Z  -  3  -  1)  270 


.i^JLkJL-ia  .U  *  a  "JtV 
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Table  6 

Analysis  of  Variance  Summary  Table 

Source _ _ _ df  SS _ MS _ 

Sequences  mn  -1*5  78.67  15.73 

Positions  m(n  -  1)  *  4  6.67  1.67 

within  squares 
(cols) 

Direct  Treatment 
and  Residual 


Treatment  effects 

n  -  1  ■ 

2 

24.33 

12.17 

(unadj ) 

Residual  effects 

n  -  1  ■ 

2 

61.00 

30.50 

10.78 

(adj) 

Residual  effects 
(unadj ) 

n  -  1  * 

2 

23.27 

11.64 

Treatment  effects 

n  -  1  - 

2 

62.07 

31.04 

10.97 

Total 


mn2  -1  ■  17 


182.0 
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Naval  Air  Station 

Mayport,  FL  32228 

Commanding  Officer 

Human  Resource  Management  Center 

Pearl  Harbor,  HI  96860 

Commander  In  Chief 
Human  Resource  Management  Division 
O.S.  Pacific  Fleet 
Pearl  Harbor,  HI  96860 

Officer  In  Charge 

Human  Resource  Management  Detachment 
Naval  Base 

Charleston,  SC  29408 

Commanding  Officer 
Human  Resource  Management  School 
Naval  Air  Station  Memphis 
Millington,  TN  38054 

Human  Resource  Management  School 
Naval  Air  Station  Memphis  (96) 
Millington,  TN  38054 


List  7  (Continued) 


Commanding  Officer 
Hunan  Resource  Management  Center 
1300  Wilson  Boulevard 
Arlington,  VA  22209 

Commanding  Officer 
Human  Resource  Management  Center 
5621-23  Tidewater  Drive 
Norfolk,  VA  23511 

Commander  In  Chief 

Human  Resource  Managenent  Division 

U.S.  Atlantic  Fleet 

Norfolk,  VA  23511 

Officer  In  Charge 

Human  Resource  Management  Detachment 
Naval  Air  Station  Whldbey  Island 
Oak  Harbor,  WA  98278 

Commanding  Officer 

Human  Resource  Management  Center 

Box  23 

FPO  New  York  09510 

Commander  In  Chief 

Human  Resource  Management  Division 

U.S.  Naval  Force  Europe 

FPO  New  York  09510 

Officer  In  Charge 

Human  Resource  Management  Detachment 
Box  60 

FPO  San  Francisco  96651 
Officer  In  Charge 

Human  Resource  Management  Detachment 

COMNAVFORJAPAN 

FPO  Seattle  98762 


4420E 
Aug  83 


LIST  8 

NAVY  MISCELLANEOUS 


Naval  Military  Personnel  Command  (2  copies) 
HRM  Department  (NMPC-6) 

Washington,  DC  20350 


Naval  Training  Analysis 
and  Evaluation  Group 
Orlando,  FL  32813 

Commanding  Officer 
ATTN:  TIC,  Bldg.  2068 
Naval  Training  Equipment  Center 
Orlando,  FL  32813 

Chief  of  Naval  Education 
and  Training  (N-5) 

Director,  Research  Development, 

Test  and  Evaluation 
Naval  Air  Station 
Pensacola,  FL  32508 

Chief  of  Naval  Technical  Training 
ATTN:  Cod-  D17 
NAS  Memphis  (75) 

Millington,  TN  38D54 

Navy  Recruiting  Command 

Head,  Research  and  Analysis  Branch 

Code  434,  Room  8001 

801  North  Randolph  Street 

Arlington,  VA  22203 

Navy  Recruiting  Command 

Director,  Recruiting  Advertising  Dept. 

Code  40 

801  North  Randolph  Street 
Arlington,  VA  22203 

Naval  Weapons  Center 
Code  094 

China  Lake,  CA  93555 
Jesse  Orlansky 

Institute  for  Defense  Analyses 
1801  North  Beauregard  Street 
Alexandria,  VA^,  22311 
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LIST  9 
USMC 


Headquarters*  U.S.  Marine  Corps 
Code  MPI-20 
Washington,  DC  20380 

Headquarters,  U.S.  Marine  Corps 
ATTN:  Scientific  Adviser, 

Code  RD-1 

Washington,  DC  20380 

Education  Advisor 
Education  Center  (E031) 

MCDEC 

Quant ico,  VA  22134 

Commanding  Officer 
Education  Center  (E031) 

MCDEC 

Quantico,  VA  22134 

Commanding  Officer 
U.S.  Marine  Corps 
Command  and  Staff  College 
Quantico,  VA  22134 


4420E 
Aug  83 


LIST  10 

OTHER  FEDERAL  GOVERNMENT 


Defense  Advanced  Research 
Projects  Agency 
Director,  Cybernetics 
Technology  Office 
1400  Wilson  Blvd,  Rn  625 
Arlington,  VA  22209 

Dr.  Douglas  Hunter 
Defense  Intelligence  School 
Washington,  DC  20374 

Dr.  Brian  Usllaner 
GAO 

Washington,  DC  20548 

National  Institute  of  Education 
E0LC/SM0 

1200  19th  Street,  N.W. 

Washington,  DC  20208 

National  Institute  of  Mental  Health 
Division  of  Extramural  Research  Programs 
5600  Fishers  Lane 
Rockville,  MD  20852 

National  Institute  of  Mental  Health 
Minority  Group  Mental  Health  Programs 
Room  7  -  102 
5600  Fishers  Lane 
Rockville,  MD  20852 

Office  of  Personnel  Management 
Office  of  Planning  and  Evaluation 
Research  Management  Division 
1900  E  Street,  N.W. 

Washington,  DC  20415 

Chief,  Psychological  Research  Branch 
U.S.  Coast  Guard  (G-P-1/2/TP42) 
Washington,  D.C.  20593 

Social  and  Developmental  Psychology 
Program 

National  Science  Foundation 
Washington,  D.C.  20550 

Dr.  Earl  Potter 

U.S.  Coast  Guard  Academy 

New  London,  CT  06320 
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LIST  10  CONT’D  ' 

OTHER  FEDERAL  GOVERNMENT 


Division  of  Industrial  Science 
&  Technological  Innovation 
Productivity  Improvement  Research 
National  Science  Foundation 
Washington,  D.C.  20550 

Douglas  B.  Blackburn,  Director 
National  Defense  University 
Mobilization  Concepts  Development 
Center 

Washington,  D.C.  20319 
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LIST  11 
ARMY 


Headquarters,  FORSCOM 

ATTH:  AFPR-HR 

Ft.  McPherson,  GA  30330 

Artsy  Research  Institute 
Field  Unit  -  Leavenworth 
P.0.  Box  3122 

Fort  Leavenworth,  XS  66027 

Technical  Director  (3  copies) 

Artsy  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Head,  Department  of  Behavior 

Science  and  Leadership 

U.S.  Military  Acadetsy,  New  York  10996 

Walter  Reid  Army  Medical  Center 
Attn:  Dr.  Mary  Lozano 
W.  R.  Army  Institute  of  Research 
Division  of  Neuropsychiatry 
Forest  Glen 

Washington,  D.C.  20012 
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LIST  12 
AIR  FORCE 


Air  University  Library 
LSE  76-443 

Maxwell  AFB,  AL  36112 

Head,  Department  of  Behavioral 

Science  and  Leadership 

U.S.  Air  Force  Academy,  CO  80840 


MAJ  Robert  Gregory 
USAFA/DFBL 

U.S.  Air  Force  Academy,  CO  80840 

AFOSR/NL 
Building  410 
Bolling  AFB 
Washington,  DC  20332 

Department  of  the  Air  Force 

HQUSAF/MPXHL 

Pentagon 

Washington,  DC  20330 
» 

Technical  Director 
AFHRL/MO(T) 

Brooks  AFB 

San  Antonio,  TX  78235 

AFMPC/MPCYPR 
Randolph  AFB,  TX  78150 
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LIST  13 
MISCELLANEOUS 


Australian  Embassy 
Office  of  the  Air  Attache  (S3B) 
1601  Massachusetts  Avenue,  N.U. 
Washington,  D.C.  20036 

British  Embassy 

Scientific  Information  Officer 
Room  509 

3100  Massachusetts  Avenue,  N.W. 
Washington,  DC  20008 

Canadian  Defense  Liaison  Staff, 
Washington 
ATTN:  CDRD 

2450  Massachusetts  Avenue,  N.W. 
Washington,  DC  20008 

Commandant,  Royal  Military 
College  of  Canada 
ATTN:  Department  of  Military 
Leadership  and  Management 
Kingston,  Ontario  K7L  2W3 

National  Defence  Headquarters 
DFAR 

Ottawa,  Ontario  K1A  OK2 

Mr.  Luigi  Petrullo 

2431  North  Edgevood  Street 

Arlington,  VA  22207 
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Sequential  by  Principal  Investigator 
LIST  14 

CURRENT  CONTRACTORS 
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Dr.  Clayton  F.  Alderfer 
Yale  University 

School  of  Organization  and  Management 
New  Haven,  Connecticut  06520 

Dr.  Janet  L.  Bames-Farrell 
Department  of  Psychology 
University  of  Hawaii 
2430  Campus  Road 
Honolulu,  HI  96822 

Dr.  Gary  Bowen 
SRA  Corporation 
800  18th  Street,  N.W. 

Washington,  D.C.  20006 

Dr.  Jomllls  Braddock 
John  Hopkins  University 
Center  for  the  Social  Organization 
of  Schools 

3505  N.  Charles  Street 
Baltimore,  MD  21218 

Jeanne  M.  Brett 
Northwestern  University 
Graduate  School  of  Management 
2001  Sheridan  Road 
Evanston,  IL  60201 

Dr.  Terry  Connolly 
Georgia  Institute  of  Technology 
School  of  Industrial  6  Systems 
Engineering 
Atlanta,  GA  30332 

Dr.  Richard  Daft 
Texas  ABM  University 
Department  of  Management 
College  Station,  TX  77843 

Dr.  Randy  Dunham 
University  of  Wisconsin 
Graduate  School  of  Business 
Madison,  WI  53706 
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List  14  (continued) 


Dr.  Henry  Emurian 
The  Johns  Hopkins  University 
School  of  Medicine 
Department  of  Psychiatry  and 
Behavioral  Science 
Baltimore,  MD  21205 

Dr.  Arthur  Gerstenfeld 
University  Faculty  Associates 
710  Commonwealth  Avenue 
Newton,  MA  02159 

Dr.  J.  Richard  Hackman 
School  of  Organization 
and  Management 
Box  1A,  Tale  University 
New  Haven,  CT  06520 

Dr.  Wayne  Holder 
American  Humane  Association 
P.0.  Box  1266 
Denver,  CO  80201 

Dr.-  Daniel  Ilgen 
Department  of  Psychology 
Michigan  State  University 
East  Lansing,  MI  48824 

Dr.  Lawrence  R.  James 
School  of  Psychology 
Georgia  Institute  of 
Technology 
Atlanta,  GA  30332 

Dr.  David  Johnson 
Professor,  Educational  Psychology 
178  Plllsbury  Drive,  S.E. 
University  of  Minnesota 
Minneapolis,  MN  55455 

Dr.  F.  Craig  Johnson 
Department  of  Educational 
Reseach 

Florida  State  University 
Tallahassee,  FL  32306 
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List  14  (continued) 


Dr.  Dan  Landis 
Department  of  Psychology 
Purdue  University 
Indianapolis,  IN  46205 

Dr.  Frank  J.  Landy 
The  Pennsylvania  State  University 
Department  of  Psychology 
417  Bruce  V.  Moore  Building 
University  Park,  PA  16802 

Dr.  Bibb  Latane 

The  University  of  North  Carolina 
at  Chapel  Hill 
Manning  Hall  026A 
Chapel  Hill,  NC  27514 

Dr.  Edward  E.  Lawler 
University  of  Southern  California 
Graduate  School  of  Business 
Administration 
Los  Angeles,  CA  90007 

Dr.  William  H.  Mobley 

College  of  Business  Administration 

Texas  A&M  University 

College  Station,  TX  77843 

Dr.  Lynn  Oppenheim 
Wharton  Applied  Research  Center 
University  of  Pennsylvania 
Philadelphia,  PA  19104 

Dr.  Thomas  M.  Ostrom 
The  Ohio  State  University 
Department  of  Psychology 
116E  Stadium 
404C  West  17th  Avenue 
Coluariros,  OH  43210 

Dr.  William  G.  Ouchl 
University  of  California, 

Los  Angeles 

Graduate  School  of  Management 
Los  Angeles,  CA  90024 
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List  14  (continued) 


Dr.  Robert  Rice 

State  University  of  New  York  at  Buffalo 
Department  of  Psychology 
Buffalo,  NY  14226 

Dr.  Irvin  G.  Sarason 
University  of  Washington 
Department  of  Psychology,  NI-25 
Seattle,  WA  98195 

Dr.  Benjamin  Schneider 
Department  of  Psychology 
University  of  Maryland 
College  Park,  MD  20742 

Dr.  Edgar  H.  Scheln 
Massachusetts  Institute  of 
Technology 

Sloan  School  of  Management 
Cambridge,  MA  02139 

Dr.  H.  Wallace  Slnaiko 
Program  Director,  Manpower  Research 
and  Advisory  Services 
Smithsonian  Institution 
801  N.  Pitt  Street,  Suite  120 
Alexandria,  VA  22314 

Dr.  Richard  M.  Steers 
Graduate  School  of  Management 
University  of  Oregon 
Eugene,  OR  97403 

Dr.  Siegfried  Streufert 
The  Pennsylvania  State  University 
Department  of  Behavioral  Science 
Milton  S.  Rershey  Medical  Center 
Hershey,  PA  17033 

Dr.  Barbara  Saboda 
Public  Applied  Systems  Division 
Westlnghouse  Electric  Corporation 
P.0.  Box  866 
Columbia,  MD  21044 

Dr.  Harry  C.  Triandls 
Department  of  Psychology 
University  of  Illinois 
Champaign,  IL  61820 
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List  14  (continued) 


Dr.  Anne  S.  Tsui 

Duke  University 

The  Fuqus  School  of  Business 

Durham,  NC  27706 

Andrew  H.  Van  de  Ven 
University  of  Minnesota 
Office  of  Research  Administration 
1919  University  Avenue 
St.  Paul,  MM  55104 

Dr.  Philip  Wexler 
University  of  Rochester 
Graduate  School  of  Education  A 
Human  Development 
Rochester,  HT  14627 

Sabra  Woolley 
SRA  Corporation 
901  South  Highland.  Street 
Arlington,  VA  22204 
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